Maximum Entropy Modeling Toolkit
نویسنده
چکیده
The Maximum Entropy Modeling Toolkit supports parameter estimation and prediction for statistical language models in the maximum entropy framework. The maximum entropy framework provides a constructive method for obtaining the unique conditional distribution p*(y|x) that satisfies a set of linear constraints and maximizes the conditional entropy H(p|f) with respect to the empirical distribution f(x). The maximum entropy distribution p*(y|x) also has a unique parametric representation in the class of exponential models, as m(y|x) = r(y|x)/Z(x) where the numerator m(y|x) = prod_i alpha_i^g_i(x,y) is a product of exponential weights, with alpha_i = exp(lambda_i), and the denominator Z(x) = sum_y r(y|x) is required to satisfy the axioms of probability. This manual explains how to build maximum entropy models for discrete domains with the Maximum Entropy Modeling Toolkit (MEMT). First we summarize the steps necessary to implement a language model using the toolkit. Next we discuss the executables provided by the toolkit and explain the file formats required by the toolkit. Finally, we review the maximum entropy framework and apply it to the problem of statistical language modeling. Keywords: statistical language models, maximum entropy, exponential models, improved iterative scaling, Markov models, triggers.
منابع مشابه
A Note on the Bivariate Maximum Entropy Modeling
Let X=(X1 ,X2 ) be a continuous random vector. Under the assumption that the marginal distributions of X1 and X2 are given, we develop models for vector X when there is partial information about the dependence structure between X1 and X2. The models which are obtained based on well-known Principle of Maximum Entropy are called the maximum entropy (ME) mo...
متن کاملModeling of the Maximum Entropy Problem as an Optimal Control Problem and its Application to Pdf Estimation of Electricity Price
In this paper, the continuous optimal control theory is used to model and solve the maximum entropy problem for a continuous random variable. The maximum entropy principle provides a method to obtain least-biased probability density function (Pdf) estimation. In this paper, to find a closed form solution for the maximum entropy problem with any number of moment constraints, the entropy is consi...
متن کاملHabitat suitability modeling of water birds and waders in Hamun wetland by by Maximum Entropy model
Climate change and human activities have increased negative pressure on natural ecosystems. Wetlands are such ecosystems that widely affected by these negative changes. Birds as a part of wildlife in a wetland have damaged by destruction of wetlands, so, a large group of them, are at risk of extinction. Habitat destruction in wetlands in arid and semi-arid areas has more negative effects on the...
متن کاملMaximum Entropy Part-of-Speech Tagging in NLTK
In this paper we implement a part of speech tagger for NLTK using maximum entropy methods. Our tagger can be used as a drop-in replacement for any of the other NLTK taggers. We give a brief tutorial on how to use our tagger as well as describing the implementation at a high level. We evaluate our tagger on the Penn Tree Bank and compare our results to those of previous work.
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره cmp-lg/9612005 شماره
صفحات -
تاریخ انتشار 1996